Coding device, imaging device, coding transmission system, and coding method

ABSTRACT

A coding device includes: an obtaining section configured to obtain moving image data including a plurality of pieces of image data reproducible at a predetermined rate; a determining section configured to determine whether coding of each piece of the image data of the moving image data obtained by the obtaining section is necessary, and output an indicating signal indicating image data set as a coding object or image data not set as a coding object; and a coding performing section configured to be supplied with the indicating signal, and code the plurality of pieces of image data obtained by the obtaining section while discretely reducing the image data according to determination of the determining section.

BACKGROUND

The present disclosure relates to a coding device and a coding methodfor moving image data including a plurality of pieces of image data.

Moving image data including a plurality of pieces of image data is codedby a coding device (Japanese Patent Laid-Open Nos. 2008-236789 and2000-050254 (hereinafter referred to as Patent Documents 1 and 2)).

The coding device codes the moving image data by a coding systemconforming to a standard such as MPEG (Moving Picture Experts Group) 1,2, or 4, H.264, or the like.

The coding device for example adjusts a quantization parameter such as aQp value or the like, inserts skip data in macroblock units, or insertsskip frame data in frame units, according to coding conditions.

The coding device thereby performs the coding process while adjustingthe data amount of coded moving image data.

SUMMARY

However, such a coding device may drop frame data according to thecoding conditions so that the coded moving image data is a predetermineddata amount.

For example, when the coding device predicts that the coded moving imagedata will exceed a predetermined transmission data amount while coding aplurality of frames of a moving image in order, the coding device skipsframe data being processed or next frame data.

In addition, in Patent Document 1, a frame to be skipped is determinedon the basis of relation to a preceding frame and a succeeding frame.

In addition, in Patent Document 2, a frame to be skipped is determinedby judgment based on a video object and the like.

When frame data not to be coded is thus determined according to codingconditions or the like, the frame data of the coded moving image data ismissing at irregular intervals.

When the moving image data having the frame data thus missing atirregular intervals is reproduced, frame intervals are not stable, andthus a condition occurs in which an image changes immediately or animage stops for a while during the reproduction.

When such a condition occurs, a viewer feels a sense of incongruityabout the reproduced moving image.

The coding device is thus desired to reduce the sense of incongruity(unnaturalness) of a viewer which sense of incongruity (unnaturalness)is attendant on the discontinuity of the coded moving image data.

According to a first embodiment of the present disclosure, there isprovided a coding device including: an obtaining section configured toobtain moving image data including a plurality of pieces of image datareproducible at a predetermined rate; a determining section configuredto determine whether coding of each piece of image data of the movingimage data obtained by the obtaining section is necessary, and output anindicating signal indicating image data set as a coding object or imagedata not set as a coding object; and a coding performing sectionconfigured to be supplied with the indicating signal, and code theplurality of pieces of image data obtained by the obtaining sectionwhile discretely reducing the image data according to determination ofthe determining section. The determining section determines whether thecoding of each piece of image data is necessary such that reproductionintervals of a plurality of pieces of image data in coded moving imagedata coded by the coding performing section are stabilized.

According to the first embodiment, the determining section determineswhether the coding of each piece of image data is necessary such thatreproduction intervals of a plurality of pieces of image data in codedmoving image data coded by the coding performing section are stabilized.

In addition, the coding performing section codes the plurality of piecesof image data obtained by the obtaining section while discretelyreducing the image data according to determination of the determiningsection.

Thus, the reproduction intervals of the plurality of pieces of imagedata in the coded moving image data are stabilized.

According to a second embodiment of the present disclosure, there isprovided an imaging device including: an imaging section configured tooutput moving image data including a plurality of pieces of image dataobtained by imaging at a predetermined rate; and a coding sectionconfigured to code the moving image data. The coding section includes anobtaining section configured to obtain the moving image data output fromthe imaging section, a determining section configured to determinewhether coding of each piece of image data of the moving image dataobtained by the obtaining section is necessary, and output an indicatingsignal indicating image data set as a coding object or image data notset as a coding object, and a coding performing section configured to besupplied with the indicating signal, and code the plurality of pieces ofimage data obtained by the obtaining section while discretely reducingthe image data according to determination of the determining section.The determining section determines whether the coding of each piece ofimage data is necessary such that reproduction intervals of a pluralityof pieces of image data in coded moving image data coded by the codingperforming section are stabilized.

According to a third embodiment of the present disclosure, there isprovided a coding transmission system including: a coding sectionconfigured to code moving image data including a plurality of pieces ofimage data obtained by imaging at a predetermined rate; a transmittingsection configured to transmit the coded moving image data coded by thecoding section; a receiving section configured to receive the codedmoving image data transmitted by the transmitting section; and adecoding section configured to decode the coded moving image datareceived by the receiving section. The coding section includes anobtaining section configured to obtain the moving image data output froman imaging section, a determining section configured to determinewhether coding of each piece of image data of the moving image dataobtained by the obtaining section is necessary, and output an indicatingsignal indicating image data set as a coding object or image data notset as a coding object, and a coding performing section configured to besupplied with the indicating signal, and code the plurality of pieces ofimage data obtained by the obtaining section while discretely reducingthe image data according to determination of the determining section.The determining section determines whether the coding of each piece ofimage data is necessary such that reproduction intervals of a pluralityof pieces of image data in coded moving image data coded by the codingperforming section are stabilized.

According to a fourth embodiment of the present disclosure, there isprovided a coding method including: an obtaining section of a codingdevice including the obtaining section, a determining section, and acoding performing section configured to code moving image data includinga plurality of pieces of image data reproducible at a predetermined rateobtaining the moving image data; the determining section determiningwhether coding of each piece of image data of the obtained moving imagedata is necessary; and the coding performing section coding the imagedata of the obtained moving image data while discretely reducing theimage data of the moving image data according to the determination. Thedetermining section determines whether the coding of each piece of imagedata is necessary such that reproduction intervals of a plurality ofpieces of image data in coded moving image data are stabilized.

According to the present disclosure, discontinuity does not occur easilyin coded moving image data, and the sense of incongruity (unnaturalness)of a viewer can be reduced.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic block diagram of a coding device according to afirst embodiment of the present disclosure;

FIG. 2 is a block diagram of a computer device for implementing thecoding device in FIG. 1;

FIG. 3 is a diagram of assistance in explaining an example of movingimage data to be coded by the coding device in FIG. 1;

FIG. 4 is a flowchart of operation of an input section in FIG. 1;

FIG. 5 is a block diagram of a video coding performing section in FIG.1;

FIG. 6 is a flowchart of a coding operation for each piece of frame databy the video coding performing section in FIG. 1;

FIG. 7 is a flowchart of coding control operation of a coding controlsection in FIG. 1;

FIG. 8 is a diagram of assistance in explaining the coding controloperation of the coding control section in FIG. 1;

FIG. 9 represents an initializing process sequence of the coding devicein FIG. 1;

FIGS. 10A and 10B are diagrams of assistance in explaining codingpresence or absence information that the coding control section in FIG.1 outputs to the video coding performing section;

FIG. 11 represents a typical coding process sequence of the inputsection and the video coding performing section in FIG. 1;

FIG. 12 represents a coding process sequence when the coding controlsection in FIG. 1 starts control;

FIG. 13 represents a coding process sequence after the coding controlsection in FIG. 1 starts control;

FIGS. 14A, 14B, and 14C are diagrams showing an example of changes indata amount of moving image data coded by the coding device in FIG. 1;

FIGS. 15A and 15B are diagrams of assistance in explaining an underflowsuppressing control operation by the coding control section when codedmoving image data underflows;

FIGS. 16A and 16B are diagrams of assistance in explaining an overflowsuppressing control operation by the coding control section when codedmoving image data overflows;

FIGS. 17A and 17B are diagrams of assistance in explaining a firstexample of moving image data coded by the coding device in FIG. 1;

FIGS. 18A and 18B are diagrams of assistance in explaining a secondexample of moving image data coded by the coding device in FIG. 1;

FIGS. 19A and 19B are diagrams of assistance in explaining a thirdexample of moving image data coded by the coding device in FIG. 1;

FIGS. 20A and 20B are diagrams of assistance in explaining a fourthexample of moving image data coded by the coding device in FIG. 1;

FIGS. 21A and 21B are diagrams of assistance in explaining a fifthexample of moving image data coded by the coding device in FIG. 1;

FIGS. 22A and 22B are diagrams of assistance in explainingpostprocessing operation by an output section in FIG. 1 (first example);

FIGS. 23A and 23B are diagrams of assistance in explainingpostprocessing operation by the output section in FIG. 1 (secondexample);

FIG. 24 is a schematic block diagram showing a coding transmissionsystem according to a second embodiment of the present disclosure; and

FIG. 25 is a diagram of assistance in explaining coding controloperation by the coding control section at a time of coding by an H.264system or an MPEG-2 system.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

Preferred embodiments of the present disclosure will hereinafter bedescribed with reference to the drawings.

Description will be made in the following order.

-   -   1. First Embodiment (Example of Coding Device)    -   2. Second Embodiment (Example of Coding Transmission System        Having Imaging device)<

1. First Embodiment Configuration of Coding Device 1

FIG. 1 is a schematic block diagram of a coding device 1 according to afirst embodiment of the present disclosure.

The coding device 1 in FIG. 1 codes input moving image data.

The coding device 1 in FIG. 1 includes an input moving image datastorage section 10, an input section 11, a coding condition storagesection 13, a coding control section 14, a video coding performingsection 15, an output section 16, and an output moving image datastorage section 17.

FIG. 2 is a block diagram of a computer device 100 for implementing thecoding device 1 in FIG. 1.

The computer device 100 in FIG. 2 has a CPU (Central Processing Unit)101, a memory (MEM) 102, and an input-output section (I/O) 103.

The memory 102 stores a coding program.

The coding program stored in the memory 102 may be stored in the memory102 before shipment of the computer device 100, or may be stored in thememory 102 after shipment of the computer device 100.

The coding program stored in the memory 102 after shipment of thecomputer device 100 may be for example obtained by reading a codingprogram recorded on a computer-readable recording medium or downloadedvia a transmission medium such as the Internet or the like.

The CPU 101 reads and executes the coding program stored in the memory102.

Thereby, the input section 11, the video coding performing section 15,and the output section 16 in FIG. 1 are implemented in the CPU 101.

In addition, the input moving image data storage section 10, the outputmoving image data storage section 17, and the coding condition storagesection 13 are implemented in the memory 102.

Incidentally, in FIG. 2, the moving image data input to the codingdevice 1 is for example input from the input-output section 103 andstored in the memory 102.

In addition, coded moving image data generated by the coding device 1 isstored in the memory 102, and thereafter output from the input-outputsection 103 to the outside.

The input moving image data storage section 10 in FIG. 1 stores inputmoving image data 21 and input data information 22.

FIG. 3 is a diagram of assistance in explaining an example of the inputmoving image data 21 to be coded by the coding device 1 in FIG. 1.

The input moving image data 21 in FIG. 3 has a plurality of pieces offrame data 23.

The frame data 23 in FIG. 3 is image data of still images reproducibleat a certain frame rate. The plurality of pieces of frame data 23 inFIG. 3 are reproduced in order from a frame on a left side, for example.

The moving image data in FIG. 3 is coded in units of three frames, forexample. A unit of this coding process is referred to as a GOP (Group OfPictures).

In the example of FIG. 3, a first frame from the left in each GOP iscoded as an I (Intra) picture. A second frame is coded as a B(Bidirectional predictive) picture. A third frame is coded as a P(Predictive) picture.

The input data information 22 is information on the input moving imagedata 21.

Such information includes for example information on the frame rate ofthe input moving image data 21 and information on a photographingenvironment in which the input moving image data 21 was taken.

In FIG. 1, the input section 11 is connected to the input moving imagedata storage section 10 and the video coding performing section 15.

The input section 11 obtains the input moving image data 21 and theinput data information 22 from the input moving image data storagesection 10.

The input section 11 outputs each piece of frame data 23 of the obtainedinput moving image data 21 to the video coding performing section 15.

At this time, the input section 11 determines output intervals of eachpiece of frame data 23 on the basis of the frame rate of the inputmoving image data 21 which frame rate is based on the input datainformation 22, and outputs the plurality of pieces of frame data 23 tothe video coding performing section 15 at the output intervals.

FIG. 4 is a flowchart of operation of the input section 11 in FIG. 1.

FIG. 4 is a flowchart in a case where the input section 11 obtains theinput moving image data 21 and the input data information 22 from theinput-output section 103 in FIG. 2, for example.

Obtaining the input moving image data 21 and the input data information22, the input section 11 stores the input moving image data 21 and theinput data information 22 in the input moving image data storage section10. In addition, the input section 11 stores coding presence or absenceinformation in the input moving image data storage section 10 (stepST1).

The coding condition storage section 13 in FIG. 1 stores initial codinginformation 24 set in advance as coding conditions for the input movingimage data 21.

Incidentally, the initial coding information 24 may be set in the codingdevice 1 in advance, or may be input from the outside via theinput-output section 103 in FIG. 2, for example.

The initial coding information 24 includes for example information suchas a profile, a level, the bit rate of coded moving image data, GOPlength, the frame intervals of the coded moving image data, and thelike, which are necessary for coding.

The GOP length indicates the number of frames (number of images)included in one GOP, for example.

The frame intervals are a recording frame rate when moving image data isrecorded, for example.

The coding control section 14 is connected to the input moving imagedata storage section 10, the coding condition storage section 13, andthe video coding performing section 15.

The coding control section 14 outputs an indicating signal to the videocoding performing section 15 on the basis of the input data information22 in the input moving image data storage section 10 and the initialcoding information 24 in the coding condition storage section 13.

For example, the coding control section 14 indicates an initial framerate for coding in the video coding performing section 15 on the basisof the frame rate of the input moving image data 21 and the frame rateof the initial coding information 24.

In addition, the coding control section 14 determines conditions ofcoding by the video coding performing section 15 on the basis of anotification signal input from the video coding performing section 15 ineach cycle of the GOP length, for example.

In addition, the coding control section 14 updates the contents of theindicating signal according to a result of the determination.

For example, the coding control section 14 compares the data amount ofthe moving image data coded by the video coding performing section 15with the data amount of a buffer 37 defined as a buffer storing thecoded moving image data, and updates the indicating signal to contentscorresponding to relation between these data amounts.

The coding control section 14 indicates whether each piece of frame data23 of the input image data is image data set as a coding object or imagedata not set as a coding object.

The video coding performing section 15 is connected to the input section11, the output section 16, the coding condition storage section 13, andthe coding control section 14.

The video coding performing section 15 codes the frame data 23 inputfrom the input section 11, and outputs the coded frame data 23 to theoutput section 16.

At this time, the video coding performing section 15 independentlydetermines whether the coding of frame data 23 is necessary or not onthe basis of the initial coding information 24 stored in the codingcondition storage section 13, the indicating signal input from thecoding control section 14, and coding conditions (the data amount of thecoded moving image data and the like).

In addition, the video coding performing section 15 independentlydetermines a frame coding system, a parameter (coding variable), and thelike.

Then, the video coding performing section 15 codes the frame data 23input from the input section 11 according to the determinations of thevideo coding performing section 15 itself, and outputs the coded framedata 23 to the output section 16.

In addition, the video coding performing section 15 outputs thenotification signal for notifying the data amount of the coded movingimage data to the coding control section 14 in each cycle correspondingto the GOP length.

FIG. 5 is a block diagram of the video coding performing section 15 inFIG. 1.

The video coding performing section 15 in FIG. 5 codes moving image databy an MPEG-4 visual system.

The video coding performing section 15 in FIG. 5 has a first framememory 31, a motion vector detector 32, a subtracter 33, a DCT (DiscreteCosine Transform) unit 34, and a quantizer 35.

The video coding performing section 15 in FIG. 5 also has a VLC(Variable Length Code) unit 36 and a buffer 37.

The video coding performing section 15 in FIG. 5 also has a dequantizer38, an IDCT (Inverse DCT) unit 39, a second frame memory 40, and amotion compensator 41.

The first frame memory 31 stores an image based on frame data 23 outputfrom the input section 11.

The motion vector detector 32 is connected to the first frame memory 31.

The motion vector detector 32 detects a motion vector of an objectcommon to an image of other frame data 23 for a new coding object imagestored in the first frame memory 31.

The subtracter 33 is connected to the first frame memory 31 and themotion compensator 41. The subtracter 33 subtracts an image stored inthe second frame memory 40 and resulting from compensation processing bythe motion compensator 41 from the image stored in the first framememory 31.

The DCT unit 34 is connected to the subtracter 33. The DCT unit 34subjects each of predetermined blocks of an image resulting from thedifferential operation by the subtracter 33 to a discrete cosinetransform.

The quantizer 35 is connected to the DCT unit 34. The quantizer 35quantizes DCT coefficients of each frequency band which coefficients areobtained by the discrete cosine transform in each block by the DCT unit34.

The VLC unit 36 is connected to the quantizer 35. The VLC unit 36subjects the DCT coefficient data quantized by the quantizer 35 tovariable length coding. The VLC unit 36 outputs the coded data to thebuffer 37.

The buffer 37 thereby stores the coded frame data 23. The coded framedata 23 is output from this buffer 37 to the output section 16 in FIG.1.

The VLC unit 36 is also connected to the motion vector detector 32. TheVLC unit 36 outputs data on a motion vector detected by the motionvector detector 32 to the buffer 37.

In addition, the DCT unit 34, the quantizer 35, and the VLC unit 36 readthe indicating signal of the coding control section 14 and the initialcoding information 24 in the coding condition storage section 13 in aseries of coding processes.

The DCT unit 34, the quantizer 35, and the VLC unit 36 adjust forexample a block size, a degree of quantization, and a coding method onthe basis of the indicating signal of the coding control section 14 andthe initial coding information 24 in the coding condition storagesection 13.

The quantizer 35 also reads the data amount of the coded frame data 23stored in the buffer 37.

The quantizer 35 determines for example whether the coding of frame data23 is necessary or not on the basis of an independent decision accordingto the data amount.

The dequantizer 38 is connected to the quantizer 35. The dequantizer 38generates DCT coefficients for the quantized frame data 23 from DCTcoefficients after being quantized by the quantizer 35.

The IDCT unit 39 is connected to the dequantizer 38. The IDCT unit 39generates an image of the quantized frame data 23 from the DCTcoefficients generated by the dequantizer 38.

The second frame memory 40 stores the image of the quantized frame data23.

The motion compensator 41 is connected to the motion vector detector 32and the second frame memory 40.

The motion compensator 41 generates an image obtained by moving theimage stored in the second frame memory 40 on the basis of a motionvector detected by the motion vector detector 32.

The output section 16 in FIG. 1 is connected to the video codingperforming section 15 and the output moving image data storage section17.

The output section 16 stores the coded frame data 23 input from thevideo coding performing section 15 in the output moving image datastorage section 17.

The output moving image data storage section 17 thereby stores the framedata 23 coded by the video coding performing section 15.

The output moving image data storage section 17 stores first output data25 and second output data 26 as moving image data obtained by coding thesame input moving image data 21 as illustrated in FIG. 3.

[Coding Operation of Coding Performing Section in FIG. 1]

FIG. 6 is a flowchart of a coding operation for each piece of frame data23 by the video coding performing section 15 in FIG. 1.

The video coding performing section 15 performs a coding process of FIG.6 each time frame data 23 is input from the input section 11 accordingto the frame rate of the input frame data 23.

In the coding process for each piece of frame data 23, the video codingperforming section 15 first determines whether to code frame data 23involved in the coding process (step ST11).

For example, when an indicating signal to an effect that the frame data23 in question is not to be coded is input from the coding controlsection 14, the video coding performing section 15 in principledetermines that the frame data 23 is not to be coded.

However, when the video coding performing section 15 reads the dataamount of coded frame data 23 from the buffer 37, and determines thatthe buffer 37 will become empty when the frame data 23 in question isnot coded, the video coding performing section 15 determines that theframe data 23 is to be coded.

In addition, for example, when an indicating signal to an effect thatthe frame data 23 in question is to be coded is input from the codingcontrol section 14, the video coding performing section 15 in principledetermines that the frame data 23 is to be coded.

However, when the video coding performing section 15 reads the dataamount of the coded frame data 23 from the buffer 37, and determinesthat the buffer 37 will become full when the frame data 23 in questionis coded, the video coding performing section 15 determines that theframe data 23 is not to be coded.

When determining that the frame data 23 is to be coded, the video codingperforming section 15 performs the process of coding the frame data 23(step ST12).

The video coding performing section 15 outputs the coded frame data 23from the buffer 37 in FIG. 5 (step ST13).

In addition, the video coding performing section 15 calculates andstores the data amount of the coded frame data 23 (step ST14).

After coding the frame data 23, the video coding performing section 15determines whether timing of processing of the frame data 23 in questionis timing of outputting a notification signal to the coding controldevice (step ST15).

Also when the frame data 23 as a coding processing object is not coded,the video coding performing section 15 determines whether timing ofprocessing of the frame data 23 in question is timing of outputting anotification signal to the coding control device (step ST15).

For example, when coding is performed according to GOPs as units ofthree frames as shown in FIG. 2, for example, the video codingperforming section 15 determines that timing of outputting anotification signal arrives when a last P-picture of each GOP is coded.The video coding performing section 15 determines that timing ofoutputting a notification signal does not arrive when other pictures arecoded.

When timing of outputting a notification signal arrives, the videocoding performing section 15 notifies the data amount of the coded framedata 23 to the coding control device (step ST16). In the case of FIG. 2,the video coding performing section 15 notifies a total data amount ofthree pieces of frame data 23 coded in each GOP to the coding controldevice when a last P-picture of each GOP composed of three frames iscoded.

The video coding performing section 15 thereby ends the coding process.In addition, when timing of outputting a notification signal has notarrived, the video coding performing section 15 ends the coding process.

[Coding Control Operation of Coding Control Section 14 in FIG. 1]

FIG. 7 is a flowchart of coding control operation of the coding controlsection 14 in FIG. 1.

The coding control section 14 performs the coding control operation ofFIG. 7 each time a notification signal is input from the video codingperforming section 15, for example. In the example of FIG. 6, the videocoding performing section 15 outputs a notification signal to the codingcontrol section 14 when the video coding performing section 15 has codeda last picture of each GOP.

In the coding control operation, the coding control section 14 firstcompares an obtained code amount with a threshold value in apredetermined buffer model 51 (step ST21).

Next, the coding control section 14 determines the ratio of frame data23 to be coded within a predetermined period on the basis of a result ofthe comparison of the code amount, and selects frame data 23 to be setas objects for coding from a plurality of pieces of frame data 23 withinthe predetermined period according to the ratio (step ST22).

Next, the coding control section 14 outputs the predetermined period andan indicating signal for indicating the selected frame data 23 set asobjects for coding to the video coding performing section 15 (stepST23). The coding control section 14 indicates for example GOP lengthfor the input moving image data 21 and frame data 23 set as codingobjects or frame data 23 not set as coding objects within a GOP inquestion.

FIG. 8 is a diagram of assistance in explaining the coding controloperation of the coding control section 14 in FIG. 1.

FIG. 8 shows the buffer model 51 conforming to an MPEG-4 visualstandard. An axis of ordinates indicates an amount of data retainedwithin the buffer model 51. Incidentally, the buffer 37 in FIG. 5corresponds to this buffer model 51.

Incidentally, the retained data refers to data coded by the video codingperforming section 15 and not processed by the output section 16 in thesubsequent stage.

A VBV (Video Buffer Verifier) in FIG. 8 is a buffer occupation amountdefined in the MPEG-4 visual standard. According to the MPEG-4 visualstandard, an amount of coded data is desired to be controlled such thatthe amount of data retained within the buffer model 51 falls within arange of 0% to 100% of a VBV value.

When the buffer occupation amount is 0%, the buffer model 51 is full,and is in an underflow state.

When the buffer occupation amount is 100%, the buffer model 51 is empty,and is in an overflow state.

When the video coding performing section 15 in FIG. 1 predicts that theamount of data retained within the buffer model 51 will become less than0% of the VBV value, for example, the video coding performing section 15performs control for reducing the data amount according to an ownjudgment of the video coding performing section 15. The video codingperforming section 15 for example stops and skips the coding of framedata 23 according to an own judgment of the video coding performingsection 15.

In addition, when the video coding performing section 15 predicts thatthe amount of data retained within the buffer model 51 will exceed 100%of the VBV value, the video coding performing section 15 performscontrol for increasing the data amount according to an own judgment ofthe video coding performing section 15. The video coding performingsection 15 for example codes frame data 23 to be skipped according to anown judgment of the video coding performing section 15.

On the other hand, the coding control section 14 controls the codingoperation of the video coding performing section 15 on the basis of anoverflow suppressing threshold value 52, a threshold value 53 of 80% ofthe VBV value, a threshold value 54 of 40% of the VBV value, and anunderflow suppressing threshold value 55.

The overflow suppressing threshold value 52 is set between the VBV valueand the threshold value 53 of 80% of the VBV value.

The underflow suppressing threshold value 55 is set between zero and thethreshold value 54 of 40% of the VBV value.

Incidentally, these values are stored as initial coding information 24in the coding condition storage section 13, for example. In addition,each value can be set as appropriate by changing the initial codinginformation 24.

When the retained data amount is within a range of 80% to 40% of the VBVvalue, the coding control section 14 does not adjust a code amount. Thecoding control section 14 outputs an indicating signal having the samecontents as an indicating signal already output to the video codingperforming section 15 to the video coding performing section 15.

When the retained data amount is not within the range of 80% to 40% ofthe VBV value, on the other hand, the coding control section 14 adjuststhe code amount. The coding control section 14 outputs an indicatingsignal having different contents from those of an indicating signalalready output to the video coding performing section 15 to the videocoding performing section 15.

Specifically, in the case of FIG. 8, when the retained data amount iswithin a range of 80% of the VBV value to the overflow suppressingthreshold value 52, the coding control section 14 adjusts the codeamount so as to decrease the code amount by one step. The coding controlsection 14 for example lengthens a predetermined period (GOP length) byone step or decreases the number of pieces or the ratio of frame data 23to be coded within a GOP by one step.

In addition, when the retained data amount is within a range of theoverflow suppressing threshold value 52 to 100% of the VBV value, thecoding control section 14 adjusts the code amount so as to decrease thecode amount by two steps. The coding control section 14 for examplelengthens the predetermined period by two steps or decreases the numberof pieces or the ratio of frame data 23 to be coded within a GOP by twosteps.

In addition, when the retained data amount is within a range of 40% ofthe VBV value to the underflow suppressing threshold value 55, thecoding control section 14 adjusts the code amount so as to increase thecode amount by one step. The coding control section 14 for exampleshortens the predetermined period by one step or increases the number ofpieces or the ratio of frame data 23 to be coded within a GOP by onestep.

In addition, when the retained data amount is within a range of theunderflow suppressing threshold value 55 to 0% of the VBV value, thecoding control section 14 adjusts the code amount so as to increase thecode amount by two steps. The coding control section 14 for exampleshortens the predetermined period by two steps or increases the numberof pieces or the ratio of frame data 23 to be coded within a GOP by twosteps.

After determining the GOP length and the frame data 23 to be codedwithin a GOP, the coding control section 14 outputs an indicatingsignal.

The coding control section 14 selects frame data 23 as coding objectswhich frame data 23 is included in the input moving image data 21 suchthat the intervals of the frame data 23 to be coded under the determinedcondition are equal intervals, and outputs the indicating signal basedon the selection.

The coding control section 14 for example selects the frame rate ofmoving image data to be coded from the frame rate of the input movingimage data 21 and divisors (or multiples) of the frame rate of the inputmoving image data 21, and selects the frame data 23 as coding objects onthe basis of the selected frame rate.

Thereby, the frame rate of moving image data coded on the basis of theinstruction of the coding control section 14 is switched between 1/60 ofa second, 1/30 of a second, 1/20 of a second, 1/15 of a second, 1/12 ofa second, ⅕ of a second, ¼ of a second, ⅓ of a second, ½ of a second,and one second, for example.

In addition, on the basis of such a control instruction, the videocoding performing section 15 selects frame data 23 to be coded, andcodes the frame data 23.

Suppose for example that the frame rate of coded moving image data whenan amount of coding is not adjusted is 1/15 of a second. In this case, aone-step decrease instruction changes the frame rate of the coded movingimage data to 1/30 of a second. A two-step decrease instruction changesthe frame rate of the coded moving image data to 1/60 of a second. Inaddition, a one-step increase instruction changes the frame rate of thecoded moving image data to 1/12 of a second. A two-step increaseinstruction changes the frame rate of the coded moving image data to ⅕of a second.

[Overall Operation of Coding Device 1 in FIG. 1]

An overall operation of the coding device 1 in FIG. 1 will next bedescribed.

A normal operation will first be described.

FIG. 9 represents an initializing process sequence of the coding device1 in FIG. 1.

In FIG. 9, when input data information 22 is stored in the input movingimage data storage section 10, the input section 11 reads the input datainformation 22.

The input section 11 determines frame intervals of input moving imagedata 21 at which frame intervals the frame data 23 of the input movingimage data 21 is output to the video coding performing section 15, andinternally retains the frame intervals (step ST31).

The coding control section 14 reads the input data information 22 fromthe input moving image data storage section 10, and internally storesthe frame intervals of the input moving image data 21 (step ST32).

The video coding performing section 15 reads initial coding information24 from the coding condition storage section 13, determines a codectype, a profile, a level, a bit rate, a frame rate, GOP length, and thelike which are necessary for coding, and prepares for coding (stepST33).

The coding control section 14 reads the initial coding information 24from the coding condition storage section 13.

The coding control section 14 calculates coding presence or absenceinformation from the frame intervals calculated from the coding framerate and the previously stored frame intervals of the input moving imagedata 21, adjusts control timing, and outputs the coding presence orabsence information and the control timing (step ST34).

The video coding performing section 15 internally stores the codingpresence or absence information and the control timing determined by thecoding control section (step ST35).

FIGS. 10A and 10B are diagrams of assistance in explaining the codingpresence or absence information that the coding control section 14 inFIG. 1 outputs to the video coding performing section 15.

A plurality of pieces of frame data 23 of the input moving image data 21in FIG. 10A have fixed frame intervals.

In the example of FIG. 10B, a plurality of pieces of frame data 23 ofmoving image data according to the coding presence or absenceinformation have fixed frame intervals obtained by discretely reducingthe input moving image data 21 alternately.

In addition, in a case of a GOP length of “3,” the period of six piecesof frame data 23 of the input moving image data 21 is a periodcorresponding to the GOP length. In addition, three pieces of frame data23 alternately selected from the six pieces of frame data 23 are coded.

FIG. 11 represents a typical coding process sequence of the inputsection 11 and the video coding performing section 15 in FIG. 1.

When input moving image data 21 is stored in the input moving image datastorage section 10, the input section 11 reads the input moving imagedata 21, and outputs the input moving image data 21 to the video codingperforming section 15. The input section 11 outputs a plurality ofpieces of unprocessed frame data 23 to the video coding performingsection 15 one by one in order at frame intervals set in advance (stepsST41-1 to ST41-4).

When the frame data 23 is input, the video coding performing section 15determines whether coding is necessary or not on the basis of the codingpresence or absence information and the like. When coding is to beperformed, the video coding performing section 15 codes the frame data23 on the basis of the coding information provided in advance, andoutputs the coded frame data 23 to the output section 16 (steps ST42-1to ST42-4).

When the coded frame data 23 is input, the output section 16 stores theframe data 23 in the output moving image data storage section 17 (stepsST43-1 to ST43-4).

The video coding performing section 15 and the output section 16 repeatthe above process each time frame data 23 is output from the inputsection 11. The output moving image data storage section 17 therebystores first output data 25 formed by a plurality of pieces of codedframe data 23.

The video coding performing section 15 independently determines whetherthe coding of each piece of frame data 23 is necessary or not as shownin FIG. 6 when performing the coding process for each piece of framedata 23.

The video coding performing section 15 compares the data amount of thecoded data (frame data 23 and the like) output to the output section 16with the data amount of the buffer model 51. The video coding performingsection 15 then adjusts a code amount according to a result of the dataamount comparison to conform to transitions of control of the buffer 37as defined by a moving image standard. The video coding performingsection 15 for example adjusts a quantization parameter such as a Qpvalue or the like, inserts skip data in macroblock units, or insertsskip frame data 23.

As a result, the video coding performing section 15 may code frame data23 or may not code the frame data 23 according to an own judgment of thevideo coding performing section 15 regardless of the instruction of thecoding control section 14 which instruction indicates that the coding ofthe frame data 23 is necessary or not.

When the video coding performing section 15 thus makes a final judgmenton whether the coding of frame data 23 is necessary or not according tothe amount of data remaining in the buffer 37, frame intervals of aplurality of pieces of coded image data in coded moving image data arenot fixed.

As a result, when the coded moving image data is reproduced, an imageimmediately changes or the image appears to be stopped for a whileduring the reproduction. The image becomes an unstable image.

The frame intervals of a plurality of pieces of coded image data of thecoded moving image data controlled by the video coding performingsection 15 repeatedly as to whether the coding of the frame data 23 isnecessary or not may become unstable.

A viewer feels a sense of incongruity about such a moving image.

FIG. 12 represents a coding process sequence when the coding controlsection 14 in FIG. 1 starts control.

A basic process flow is similar to that of FIG. 11, and steps in which asimilar process is performed are identified by the same referencenumerals.

Each time frame data 23 is input, the video coding performing section 15calculates a code amount after coding the frame data 23 in question inthe buffer model 51 conforming to a standard, and internally retains thecode amount (steps ST42-1 to ST42-4).

When timing of an end of coding each GOP arrives, the video codingperforming section 15 outputs a notification signal including theinternally retained code amount after the coding to the coding controlsection 14 (step ST44).

When the notification signal is input, the coding control section 14compares the notified obtained code amount with each threshold value inthe buffer model 51 in FIG. 8 (step ST45). In addition, the codingcontrol section 14 generates coding presence or absence information foreach piece of frame data 23 according to a result of the comparison, andoutputs the coding presence or absence information as an indicatingsignal to the video coding performing section 15 (step ST46).

The video coding performing section 15 internally retains the new codingpresence or absence information received (step ST47).

FIG. 13 represents a coding process sequence after the coding controlsection 14 in FIG. 1 starts control.

A basic process flow is similar to that of FIG. 11, and steps in which asimilar process is performed are identified by the same referencenumerals.

However, the video coding performing section 15 determines whether thecoding of each piece of frame data 23 is necessary or not on the basisof the new coding presence or absence information indicated in FIG. 12,and performs a coding process.

Thus, when an instruction to code one piece of frame data out of everytwo pieces of frame data is given, for example, the video codingperforming section 15 does not output coded frame data to the outputsection 16 as shown in steps ST42-2 and ST42-4 in FIG. 13.

[Example of Coded Image Data]

FIGS. 14A, 14B, and 14C are diagrams showing an example of changes indata amount of moving image data coded by the coding device 1 in FIG. 1.

FIG. 14A is a diagram showing input moving image data 21. FIG. 14B is adiagram showing coded moving image data. FIG. 14C is a diagram showingan MPEG-4 visual buffer occupation amount. In FIGS. 14A to 14C, timepasses from a left to a right.

In FIGS. 14A to 14C, all pieces of frame data 23 of the input movingimage data 21 are coded at first.

However, as shown in FIG. 14C, the buffer occupation amount changestoward an underflow.

In this case, the coding control section 14 updates the indicatingsignal in timing T1. The video coding performing section 15 therebyswitches so as to code every second piece of frame data 23 of the inputmoving image data 21. The video coding performing section 15 codes onepiece of frame data in each period of 2V.

In the case of FIGS. 14A to 14C, the buffer occupation amount changestoward an underflow also after the first instruction change.

In this case, the coding control section 14 further updates theindicating signal in timing T2. The video coding performing section 15thereby switches so as to code every third piece of frame data 23 of theinput moving image data 21. The video coding performing section 15 codesone piece of frame data in each period of 3V.

FIGS. 15A and 15B are diagrams of assistance in explaining an underflowsuppressing control operation by the coding control section 14 whencoded moving image data underflows.

FIG. 15A is a diagram showing a buffer occupation amount. FIG. 15B is anenlarged view of a period of T3 to T4 in FIG. 15A.

In FIG. 15A, the VBV occupation amount becomes less than an underflowsuppressing threshold value 55 in timing T3. In this case, as shown inFIG. 15B, frame intervals of frame data 23 as coding objects are changedto frame intervals of every third piece of frame data 23.

As a result, the VBV occupation amount is prevented from being furtherdecreased after timing T3. That is, the VBV occupation amount isprevented from being decreased to a level estimated to be an underflowlevel by the video coding performing section 15.

FIGS. 16A and 16B are diagrams of assistance in explaining an overflowsuppressing control operation by the coding control section 14 whencoded moving image data overflows.

FIG. 16A is a diagram showing a buffer occupation amount. FIG. 16B is anenlarged view of a period of T5 to T6 in FIG. 16A.

In FIG. 16A, the VBV occupation amount exceeds an overflow suppressingthreshold value 52 in timing T7. In this case, as shown in FIG. 16B,frame intervals of frame data 23 as coding objects are changed fromframe intervals of every second piece of frame data 23 to frameintervals of each piece of frame data 23.

As a result, the VBV occupation amount is prevented from being furtherincreased after timing T7. That is, the VBV occupation amount isprevented from being increased to a level estimated to be an overflowlevel by the video coding performing section 15.

FIGS. 17A to 21B are diagrams of assistance in explaining a firstexample of moving image data coded by the coding device 1 in FIG. 1.

FIGS. 17A to 21A are diagrams showing input moving image data 21. FIGS.17B to 21B are diagrams showing coded moving image data. In the figures,V denotes a frame period of the input moving image data 21.

FIGS. 17A and 17B represent an example in which the frame rate of codedmoving image data is changed while the GOP length of the coded movingimage data remains fixed at three.

The coding control section 14 gives an instruction to perform theoperation of FIGS. 17A and 17B when a coding operation frame rate isequal to or lower than an initial setting frame rate. FIGS. 17A and 17Brepresent an example of control being performed with the set GOP lengthconverted to the number of frames within a GOP.

Specifically, on the basis of an indicating signal in timing T11, theoperation of coding frame data 23 by the video coding performing section15 is changed from a process of coding each piece of frame data 23 to aprocess of coding every second piece of frame data 23 while the GOPlength is maintained at three.

In addition, on the basis of an indicating signal in timing T12, theoperation of coding frame data 23 by the video coding performing section15 is changed from the process of coding every second piece of framedata 23 to a process of coding every third piece of frame data 23 whilethe GOP length is maintained at three. The reproduction intervals of aplurality of coded image data are thereby widened.

FIGS. 18A and 18B represent another example in which the frame rate ofcoded moving image data is changed while the GOP length of the codedmoving image data remains fixed at three. The coding control section 14gives an instruction to perform the operation of FIGS. 18A and 18B whenthe operation frame rate exceeds the initial setting frame rate. FIGS.18A and 18B represent an example of control being performed with the setGOP length converted to the number of frames within a GOP.

Specifically, on the basis of an indicating signal in timing T21, theoperation of coding frame data 23 by the video coding performing section15 is changed from the process of coding every second piece of framedata 23 to the process of coding each piece of frame data 23 while theGOP length is maintained at three. The reproduction intervals of aplurality of coded image data are thereby narrowed.

FIGS. 19A and 19B represent another example of a frame rate changeindicated by the coding control section 14 when the operation frame rateexceeds the initial setting frame rate. FIGS. 19A and 19B represent anexample of control being performed with the set GOP length converted totime.

Specifically, on the basis of an indicating signal in timing T22, theoperation of coding frame data 23 by the video coding performing section15 is changed from the process of coding every second piece of framedata 23 to the process of coding each piece of frame data 23 while theperiod of each GOP for the input moving image data 21 is maintained.

FIGS. 20A and 20B represent another example in which the frame rate ofcoded moving image data is changed while the GOP length of the codedmoving image data remains fixed at three. The coding control section 14gives an instruction to perform the operation of FIGS. 20A and 20B whenthe operation frame rate exceeds the initial setting frame rate. FIGS.20A and 20B represent an example of control being performed with the setGOP length converted to the number of frames within a GOP.

Specifically, on the basis of an indicating signal in timing T31, theoperation of coding frame data 23 by the video coding performing section15 is changed from the process of coding every second piece of framedata 23 to the process of coding each piece of frame data 23 while theGOP length is maintained at three.

In addition, the video coding performing section 15 changes from controlfor coding three pictures within a GOP into an I-picture, a P-picture,and a P-picture to control for coding three pictures within a GOP intoan I-picture, a B-picture, and a P-picture.

FIGS. 21A and 21B represent another example of a frame rate changeindicated by the coding control section 14 when the operation frame rateexceeds the initial setting frame rate. FIGS. 21A and 21B represent anexample of control being performed with the set GOP length converted totime.

Specifically, on the basis of an indicating signal in timing T32, theoperation of coding frame data 23 by the video coding performing section15 is changed from the process of coding every second piece of framedata 23 to the process of coding each piece of frame data 23 while theperiod of each GOP for the input moving image data 21 is maintained.

In addition, the video coding performing section 15 changes from controlfor coding three pictures within a GOP into an I-picture, a P-picture,and a P-picture to control for coding six pictures within a GOP into anI-picture, a B-picture, a P-picture, a B-picture, a P-picture, and aB-picture.

[Postprocessing Operation]

Description will next be made of operation of the output section 16 towhich coded video data is input from the video coding performing section15.

The output section 16 stores the coded frame data 23 input from thevideo coding performing section 15 in the output moving image datastorage section 17.

When an actual operation frame rate for coding by the video codingperforming section 15 is equal to or higher than an initial settingframe rate, in particular, the output section 16 stores first outputdata 25 and second output data 26 in the output moving image datastorage section 17.

FIGS. 22A and 22B are diagrams of assistance in explaining an example ofpostprocessing operation by the output section 16 in FIG. 1.

FIG. 22A is a diagram showing first output data 25. FIG. 22B is adiagram showing second output data 26.

In the case of FIGS. 22A and 22B, the output section 16 interprets theset GOP length as the number of frames, and stores the first output data25 and the second output data 26 in the output moving image data storagesection 17.

Specifically, the output section 16 interprets the set GOP length as thenumber of frames for each GOP in FIG. 20B, and stores the first outputdata 25 in the output moving image data storage section 17.

In addition, the output section 16 stores all pictures of each GOP inFIG. 20B as second output data 26 in the output moving image datastorage section 17.

The output section 16 thus generates output data corresponding to theinitial setting frame rate and output data corresponding to theoperation frame rate of the video coding performing section 15 on thebasis of one piece of frame data 23 coded by the video coding performingsection 15.

FIGS. 23A and 23B are diagrams of assistance in explaining anotherexample of postprocessing operation by the output section 16 in FIG. 1.

FIG. 23A is a diagram showing first output data 25. FIG. 23B is adiagram showing second output data 26.

In the case of FIGS. 23A and 23B, the output section 16 interprets theset GOP length as the number of frames, and stores the first output data25 and the second output data 26 in the output moving image data storagesection 17.

Specifically, the output section 16 interprets the set GOP length as thenumber of frames for each GOP in FIG. 21B, and stores the first outputdata 25 in the output moving image data storage section 17.

In addition, the output section 16 stores all pictures of each GOP inFIG. 21B as second output data 26 in the output moving image datastorage section 17.

As described above, in the first embodiment, the coding control section14 changes the frame rate for coding so as to stabilize the data amountof coded image data before the video coding performing section 15determines whether the coding of frame data 23 is necessary or not onthe basis of the buffer model 51.

Thus, in the first embodiment, the video coding performing section 15less frequently determines during coding whether the coding is necessaryor not according to coding conditions, and the frame rate of a pluralityof pieces of image data in coded moving image data is stabilized.

As a result, when moving image data coded by the coding device 1according to the first embodiment is reproduced, the moving imageappears to be smooth because the frame rate of the moving image data isstable.

A tendency for an image to be changed immediately or displayed for awhile during reproduction as in a case where the video coding performingsection 15 determines during coding whether the coding is necessary ornot according to coding conditions, for example, is reduced. It ispossible to prevent a sense of incongruity from being given to a viewer.

In the first embodiment, because the frame intervals of coded frame data23 are stabilized to be substantially fixed intervals, a reproducingdevice 63 for reproducing the frame data 23 can interpolate andreconstruct discretely reduced frames by a simple complementaryfunction.

In particular, the coding control section 14 selects a coding frame rateindicated to the video coding performing section 15 from the frame rateof the input moving image data 21 and divisors (or multiples) of theframe rate of the input moving image data 21.

As a result, in the first embodiment, a plurality of pieces of codedimage data correspond to image data of the input moving image data 21 ateach fixed interval. It is therefore possible to prevent a sense ofincongruity from being given to a viewer.

The coding control section 14 determines the coding frame rate indicatedto the video coding performing section 15, sharing the buffer model 51for the video coding performing section 15 to determine whether thecoding of frame data 23 is necessary or not. Thus, the control of thecoding control section 14 can prevent the video coding performingsection 15 from determining whether to add or delete a frameindependently.

In the first embodiment, for overflow suppression, frame intervals areshortened as much as possible, and the number of times of coding perunit time is increased. As a result, the first embodiment can suppress acode amount failure or the like, and makes it possible to maintain fixedframe intervals.

In the first embodiment, the coding control section 14 uses dedicatedthreshold values for overflow suppression and underflow suppression forthe same buffer model 51 (code amount) as in the video coding performingsection 15. As a result, in the first embodiment, even when there is asign of an overflow or an underflow, it is possible to return from thatstate to a normal state in an early stage. In addition, the firstembodiment can therefore suppress a skip due to a code amount failure orthe like and maintain fixed frame intervals.

In the first embodiment, when frame intervals are shortened as much aspossible, and the number of times of coding per unit time is increased,frames as an increase over an initial rate are coded as non-referenceinter-frames, as shown in FIGS. 20A and 20B and FIGS. 21A and 21B. Inaddition, the output section 16 generates first output data 25 for theinitial setting frame rate and second output data 26 for a frame ratehigher than the initial setting frame rate. As a result, a reproducingdevice 63 or the like can select output data according to theperformance of the reproducing device 63 or the like from the movingimage data of an image quality at the initial setting frame rate and themoving image data of higher image quality, and reproduce the outputdata.

When an actual operation frame rate exceeds the initial setting framerate, a time represented by one GOP can be shortened by interpreting theGOP length as the number of coded frames, as shown in FIGS. 18A and 18Band FIGS. 20A and 20B. As a result, the number of intra-frames generatedper unit time can be increased, and thus recovery points can beincreased in number.

When an actual operation frame rate exceeds the initial setting framerate, a time represented by one GOP can be lengthened by interpretingthe GOP length as coding time, as shown in FIGS. 19A and 19B and FIGS.21A and 21B. As a result, it is possible to decrease the number ofintra-frames, increase a code amount assignable to each frame, and thusimprove the image quality of each image.

When an actual operation frame rate is lower than the initial settingframe rate, a time represented by one GOP can be lengthened byinterpreting the GOP length as coding time, as shown in FIGS. 17A and17B. As a result, it is possible to decrease the number of intra-frames,increase a code amount assignable to each frame, and thus improve theimage quality of each image. In addition, a code amount (buffer model51) failure can be further suppressed due to this effect. In addition,because the number of frames carried for the length of one GOP is fixed,a code amount for one GOP can be predicted easily.

2. Second Embodiment

FIG. 24 is a schematic block diagram showing a coding transmissionsystem 61 according to a second embodiment of the present disclosure.

The coding transmission system 61 in FIG. 24 has an imaging device 62and a reproducing device 63.

The imaging device 62 has an imaging section 71, an input buffer memory72, a coding section 73, an output buffer memory 74, and a firstcommunicating section 75.

The reproducing device 63 has a second communicating section 81, areceiving buffer memory 82, a decoding section 83, a decoding buffermemory 84, a reproducing section 85, and a monitor 86.

The imaging section 71 in the imaging device 62 is for example a CMOSsensor or a CCD sensor. The imaging section 71 is connected to the inputbuffer memory 72. The imaging section 71 outputs a signal of a picked-upimage to the input buffer memory 72. The input buffer memory 72 storesone piece of picked-up image data.

The coding section 73 is the coding device 1 shown in FIG. 1. The codingsection 73 is connected to the input buffer memory 72 and the outputbuffer memory 74. The coding section 73 obtains image data (frame data23) constituting the picked-up moving image data from the input buffermemory 72, codes the image data, and outputs the coded image data to theoutput buffer memory 74. The output buffer memory 74 stores the codedimage data.

The first communicating section 75 is for example a radio communicatingsection. The first communicating section 75 is connected to the outputbuffer memory 74. The first communicating section 75 transmits the datastored in the output buffer memory 74 by radio in a predeterminedcommunication band.

The second communicating section 81 in the reproducing device 63 is aradio communicating section capable of communicating with the firstcommunicating section 75. The second communicating section 81 isconnected to the receiving buffer memory 82. The second communicatingsection 81 stores the data received from the first communicating section75 in the receiving buffer memory 82. The receiving buffer memory 82thereby stores the coded image data.

The decoding section 83 is connected to the receiving buffer memory 82and the decoding buffer memory 84. The decoding section 83 decodes thecoded image data stored in the receiving buffer memory 82, and storesthe decoded image data in the decoding buffer memory 84. The decodingbuffer memory 84 stores the decoded image data.

The reproducing section 85 is connected to the decoding buffer memory 84and the monitor 86. The reproducing section 85 outputs the image datastored in the decoding buffer memory 84 to the monitor 86 at apredetermined frame rate. The monitor 86 thereby displays a moving imageformed by a plurality of decoded images.

Each of the above embodiments is an example of a preferred embodiment ofthe present disclosure. However, the present disclosure is not limitedto this. Various modifications and changes can be made without departingfrom the spirit of the disclosure.

For example, the coding device 1 according to the foregoing firstembodiment and the coding section 73 according to the second embodimentcode each piece of frame data 23 of moving image data by the MPEG-4visual system.

In addition to this, for example, the coding device 1 or the codingsection 73 may code each piece of frame data 23 of moving image data byan H.264 system or an MPEG-2 system.

FIG. 25 is a diagram of assistance in explaining coding controloperation by the coding control section 14 at a time of coding by theH.264 system.

FIG. 25 shows a buffer model 51 conforming to the standard of the H.264system. An axis of ordinates indicates an amount of data retained withinthe buffer model 51. An axis of abscissas indicates time. Incidentally,the MPEG-2 system also uses a buffer model 51 similar to that of theH.264 system, and controls an amount of data to be coded as appropriateaccording to the amount of data retained in the buffer model 51.

In the MPEG-4 visual system, as shown in FIG. 14C, an initial value at atime of starting a coding process is set at a predetermined percentage(%) of the VBV value.

On the other hand, in the H.264 system, as shown in FIG. 25, an initialvalue at a time of starting coding is set at 0% of the VBV value.

There is thus a small buffer occupation amount immediately after a startof a coding process.

In addition, in the H.264 system, the video coding performing section 15starts coding control operation according to the data amount of codeddata for example after a predetermined period after starting the codingprocess.

Therefore, in a case where the present disclosure is applied to thevideo coding performing section 15 of the H.264 system, it suffices forthe coding control section 14 to start coding control operationaccording to the data amount of coded data after the predeterminedperiod.

Incidentally, also in MPEG-2, coding control operation according to thedata amount of coded data can be similarly started by control of thecoding control section 14.

The foregoing second embodiment is an example in which the coding device1 according to the first embodiment is applied to the imaging device 62.

The imaging device 62 includes a DSC (Digital Still Camera), a videocamera, a surveillance camera, and the like.

The coding device 1 is also applicable to for example portabletelephones, PDAs (Personal Digital Assistants), electronic book devices,notebook computer devices, navigation devices, and portable reproducingdevices.

In addition, the coding device 1 is applicable to reproducing devices,AV (Audio Visual) devices, DLNA (Digital Living Network Alliance) serverdevices, and the like.

The present disclosure contains subject matter related to that disclosedin Japanese Priority Patent Application JP 2010-146434 filed in theJapan Patent Office on Jun. 28, 2010, the entire contents of which ishereby incorporated by reference.

It should be understood by those skilled in the art that variousmodifications, combinations, sub-combinations and alterations may occurdepending on design requirements and other factors insofar as they arewithin the scope of the appended claims or the equivalents thereof.

1. A coding device comprising: an obtaining section configured to obtain moving image data including a plurality of pieces of image data reproducible at a predetermined rate; a determining section configured to determine whether coding of each piece of said image data of said moving image data obtained by said obtaining section is necessary, and output an indicating signal indicating image data set as a coding object or image data not set as a coding object; and a coding performing section configured to be supplied with said indicating signal, and code the plurality of pieces of image data obtained by said obtaining section while discretely reducing the image data according to determination of said determining section, wherein said determining section determines whether the coding of each piece of image data is necessary such that reproduction intervals of a plurality of pieces of image data in coded moving image data coded by said coding performing section are stabilized.
 2. The coding device according to claim 1, wherein said coding performing section outputs a notification signal for repeatedly notifying a data amount of said coded moving image data during performance of coding to said determining section, and said determining section outputting said indicating signal updates said indicating signal during the performance of said coding such that the reproduction intervals of the plurality of pieces of image data in the coded moving image data tend to equal intervals under a condition of the data amount of said coded moving image data, the data amount of said coded moving image data being notified repeatedly by the notification signal.
 3. The coding device according to claim 2, wherein said determining section compares the data amount of said coded moving image data, the data amount of said coded moving image data being notified by said notification signal, with a data amount of a buffer defined as a buffer storing said coded moving image data, said determining section updates said indicating signal so as to widen the reproduction intervals of the coded image data on a basis of said indicating signal when a difference between a maximum data amount of said buffer and the data amount of said coded moving image data is small, and said determining section updates said indicating signal so as to narrow the reproduction intervals of the coded image data on a basis of said indicating signal when a difference between a minimum data amount of said buffer and the data amount of said coded moving image data is small.
 4. The coding device according to claim 1, wherein said coding performing section discretely reduces and codes said image data obtained by said obtaining section so that the data amount of said coded moving image data is within a predetermined range according to an own judgment of said coding performing section on a basis of the data amount of said coded moving image data.
 5. The coding device according to claim 4, wherein said coding performing section discretely reduces the image data to be coded according to an own judgment of said coding performing section when there is no difference between a maximum data amount of a buffer defined as a buffer storing said coded moving image data and the data amount of said coded moving image data, and said coding performing section adds image data to be coded according to an own judgment of said coding performing section when there is no difference between a minimum data amount of said buffer and the data amount of said coded moving image data.
 6. The coding device according to claim 5, wherein said coding performing section determines addition or deletion of image data to be coded on a basis of whether said buffer overflows or underflows, and said determining section determines whether the coding of each piece of image data indicated in said indicating signal is necessary on a basis of a reference value set for a data amount of said buffer.
 7. The coding device according to claim 1, wherein when a coding rate of the plurality of pieces of image data coded by said coding performing section on a basis of said indicating signal is lower than a coding rate set in advance, said determining section updates said indicating signal so that the plurality of pieces of image data are coded at intervals wider than intervals corresponding to said coding rate set in advance.
 8. The coding device according to claim 1, wherein when a coding rate of the plurality of pieces of image data coded by said coding performing section on a basis of said indicating signal is higher than a coding rate set in advance, said determining section updates said indicating signal so that the plurality of pieces of image data are coded at intervals narrower than intervals corresponding to said coding rate set in advance.
 9. The coding device according to claim 8, wherein said determining section updates said indicating signal until the plurality of pieces of image data coded by said coding performing section on the basis of said indicating signal are in one-to-one correspondence with the plurality of pieces of image data obtained by said obtaining section at a maximum, and said determining section sets a maximum coding rate of the plurality of pieces of image data coded by said coding performing section equal to a rate of the plurality of pieces of image data obtained by said obtaining section.
 10. The coding device according to claim 1, wherein said determining section outputs said indicating signal for each group of a plurality of pieces of image data of the moving image data obtained by said obtaining section, and said determining section indicates whether coding of said plurality of pieces of image data included in the group in question corresponding to said indicating signal is necessary in said indicating signal.
 11. The coding device according to claim 10, wherein said determining section changes said indicating signal so as to increase or decrease a number of a plurality of pieces of image data in the moving image data obtained by said obtaining section, the number of the plurality of pieces of image data corresponding to said group, without increasing or decreasing a number of pieces of image data coded within said group according to a data amount of said coded moving image data coded by said coding performing section.
 12. The coding device according to claim 11, wherein said coding performing section outputs a notification signal for notifying a data amount of said coded moving image data to said determining section each time said coding performing section codes a plurality of pieces of image data within said group on a basis of said indicating signal, and said determining section outputting said indicating signal updates said indicating signal for a group indicated next such that the reproduction intervals of the plurality of pieces of image data in the coded moving image data tend to equal intervals under a condition of the data amount of said coded moving image data, the data amount of said coded moving image data being notified by the notification signal.
 13. The coding device according to claim 10, wherein said determining section changes said indicating signal so as to increase or decrease a number of pieces of image data coded within said group without increasing or decreasing a number of a plurality of pieces of image data in the moving image data obtained by said obtaining section, the number of the plurality of pieces of image data corresponding to said group, according to a data amount of said coded moving image data coded by said coding performing section.
 14. The coding device according to claim 13, wherein said coding performing section outputs a notification signal for notifying a data amount of said coded moving image data to said determining section each time said coding performing section codes a plurality of pieces of image data within said group on a basis of said indicating signal, and said determining section outputting said indicating signal updates said indicating signal for a group indicated next such that the reproduction intervals of the plurality of pieces of image data in the coded moving image data tend to equal intervals under a condition of the data amount of said coded moving image data, the data amount of said coded moving image data being notified by the notification signal.
 15. The coding device according to claim 1, further comprising a postprocessing section supplied with said coded moving image data from said coding performing section, wherein when a rate of the plurality of pieces of image data in said coded moving image data is higher than a coding rate set in advance, said postprocessing section generates moving image data for the higher rate and moving image data for said coding rate set in advance.
 16. An imaging device comprising: an imaging section configured to output moving image data including a plurality of pieces of image data obtained by imaging at a predetermined rate; and a coding section configured to code said moving image data, wherein said coding section includes an obtaining section configured to obtain said moving image data output from said imaging section, a determining section configured to determine whether coding of each piece of said image data of said moving image data obtained by said obtaining section is necessary, and output an indicating signal indicating image data set as a coding object or image data not set as a coding object, and a coding performing section configured to be supplied with said indicating signal, and code the plurality of pieces of image data obtained by said obtaining section while discretely reducing the image data according to determination of said determining section, and said determining section determines whether the coding of each piece of image data is necessary such that reproduction intervals of a plurality of pieces of image data in coded moving image data coded by said coding performing section are stabilized.
 17. A coding transmission system comprising: a coding section configured to code moving image data including a plurality of pieces of image data obtained by imaging at a predetermined rate; a transmitting section configured to transmit the coded moving image data coded by said coding section; a receiving section configured to receive the coded moving image data transmitted by said transmitting section; and a decoding section configured to decode the coded moving image data received by said receiving section, wherein said coding section includes an obtaining section configured to obtain said moving image data output from an imaging section, a determining section configured to determine whether coding of each piece of said image data of said moving image data obtained by said obtaining section is necessary, and output an indicating signal indicating image data set as a coding object or image data not set as a coding object, and a coding performing section configured to be supplied with said indicating signal, and code the plurality of pieces of image data obtained by said obtaining section while discretely reducing the image data according to determination of said determining section, and said determining section determines whether the coding of each piece of image data is necessary such that reproduction intervals of a plurality of pieces of image data in coded moving image data coded by said coding performing section are stabilized.
 18. A coding method comprising: an obtaining section of a coding device including said obtaining section, a determining section, and a coding performing section configured to code moving image data including a plurality of pieces of image data reproducible at a predetermined rate obtaining said moving image data; said determining section determining whether coding of each piece of said image data of the obtained said moving image data is necessary; and said coding performing section coding the image data of the obtained said moving image data while discretely reducing the image data of said moving image data according to said determination, wherein said determining section determines whether the coding of each piece of image data is necessary such that reproduction intervals of a plurality of pieces of image data in coded moving image data are stabilized. 